Room information inferring apparatus including a person detector and a presence map generator, room information inferring method including person detection and presence map generation, and air conditioning apparatus

ABSTRACT

A room information inferring apparatus that infers information regarding a room has an imaging unit that captures an image of a room that is to be subjected to inferring, a person detector that detects a person in an image captured by the imaging unit, and acquires a position of the person in the room, a presence map generator that generates a presence map indicating a distribution of detection points corresponding to persons detected in a plurality of images captured at different times, and an inferring unit that infers information regarding the room based on the presence map.

BACKGROUND

Field

The present invention relates to a technique for inferring informationregarding a room.

Related Art

With household appliances used indoors, there are cases where it may bepreferable to perform control in accordance with the shape or the likeof the room in which the household appliance is installed. For example,if an air conditioning apparatus does not perform air conditioningcontrol in accordance with the shape of the room, it is possible for airto not circulate, and for hot spots to form.

In recent years, there have been air conditioners that allow a user toinput the shape of the room, and perform air conditioning control inaccordance with the room shape that was input (see JP 2000-346432A, forexample). However, it is troublesome for the user if it is necessaryduring apparatus installation to input information such as whether theshape of the room is elongated lengthwise, elongated widthwise, square,or the like, whether the air conditioner is installed at a central,rightward, or leftward position, and whether the distance to theopposing wall is long, normal, or short. Also, since air conditionershave poor user interfaces, only a rough room shape can be input.Accordingly, there are situations in which the distance to the opposingwall is not known even if the rough room shape is known. Furthermore, ifthe shape of the actual room is a more complex shape, or if furniture isplaced in the room, it may be preferable to perform air conditioningcontrol that takes these facts into consideration, but it is difficultto realize optimal air conditioning control with merely the input of arough room shape.

Also, there is known to be technology in which a TOF ranging sensor orstereo camera ranging sensor is used to acquire the shape of a room (seeJP 2008-261567A, for example). The use of a ranging sensor makes itpossible to accurately obtain the shape of the room, thus making itpossible to perform optimal air conditioning control in accordance withthe room shape. However, a TOF ranging sensor is expensive. Also, astereo camera ranging sensor requires two cameras, and thus has a highercost than a monocular camera.

There is also known to be technology in which air conditioning controlis performed based on the presence of people in a room, the distributionof such people, and the like (see JP 2012-17936A, for example). JP2012-17936A discloses that peoples' heads are recognized, informationsuch as the number of people and distribution thereof over time in theroom is acquired, and air conditioner air conditioning control and thelike are performed based on such information. However, in the techniquedisclosed in JP 2012-17936A, the shape of the room cannot be acquired,and therefore it is not possible to perform air conditioning control inaccordance with the shape of the room.

Although the above description mainly takes the example of an airconditional apparatus, and describes that the shape of the room in whichthe apparatus is installed is used in control, the information useful inair conditioning control is not limited to the shape of the room. Otherroom-related information is also useful in air conditioning control,such as the arrangement of furniture in the room, and regions wherepeople are present in the room. Also, the apparatus to which roominformation is useful is not limited to an air conditioning apparatus,and in the case of a lighting control apparatus for example, moreappropriate lighting control can be realized by taking the shape of theroom and the like into consideration.

JP 2000-346432A, JP 2008-261567A, and JP 2012-17936A are examples ofbackground art.

SUMMARY

One or more embodiments of the present invention provides a technique inwhich information regarding a room can be inferred easily and precisely.

One or more embodiments of the present invention employs a configurationin which an image of a room is captured by an imaging unit, a person isdetected in the captured image, and information regarding the room isinferred based on a history of person detection positions.

Specifically, a room information inferring apparatus according to one ormore embodiments of the present invention is a room informationinferring apparatus that infers information regarding a room, the roominformation inferring apparatus including: an imaging unit that capturesan image of a room that is to be subjected to inferring; a persondetection unit that detects a person in an image captured by the imagingunit, and acquires a position of the person in the room; a presence mapgeneration unit that generates a presence map indicating a distributionof detection points corresponding to persons detected in a plurality ofimages captured at different times; and an inferring unit that infersinformation regarding the room based on the presence map.

Examples of room-related information include the shape of the room,furniture placement regions in the room, and people containing regions.According to this configuration, even if the user does not inputinformation, it is possible to accumulate detected positions of personsdetected in captured images, and infer information regarding the roombased on the distribution of detection points. In particular, the moreperson detection results that are accumulated, the more accuratelyinformation regarding the room can be inferred.

Person detection can be performed using various techniques. For example,the person detection unit may detect a face, a head, or an upper body ofthe person in the image, and acquire the position of the person in theroom based on a position and a size of the face, the head, or the upperbody in the image. Based on the position and size of the face, head, orupper body in the image, it is possible to determine not only ahorizontal position in the room, but also a height position. Since aface, head, or upper body is detected in the captured image in thistechnique, a normal monocular camera can be employed as the imagingunit, and it is possible to suppress the manufacturing cost. Also, bytracking the detected face or head, it is also possible to performperson tracking processing, and it is possible to acquire movement pathsand determine whether a person is moving or stationary. The use of suchinformation makes it possible to also acquire more detailed informationabout the room.

In one or more embodiments of the present invention, the inferring unitmay infer a shape of the room based on the presence map, as informationregarding the room. Processing for inferring the shape of the room basedon the presence map can be performed as follows, for example.Specifically, the inferring unit may infer that a polygon circumscribedaround the distribution of detection points in the presence map is theshape of the room. Here, the polygon may be any polygon, but it isthought that a rectangle will be employed, for example. Here, theinferring unit may infer the shape of the room based on an assumptionthat the room is defined by straight lines that are parallel in twomutually orthogonal directions. This is because the walls of a roomgenerally extend in mutually orthogonal directions. Also, according toone or more embodiments of the present invention, the two directions arerespectively an imaging direction of the imaging unit and a directionorthogonal to the imaging direction. This is because if the imaging unitis installed on a wall surface, and the direction orthogonal to thatwall surface is considered to be the imaging direction, these twodirections are the directions in which the walls extend.

Also, one or more embodiments of the present invention, when inferringthe shape of the room, the inferring unit deems that a wall surface isoutward by a predetermined distance from a shape obtained as a polygoncircumscribed around the distribution of detection points in thepresence map. This is because users are generally not located directlynext to wall surfaces, but rather move between positions separated fromwall surfaces by a predetermined distance. Note that in the case ofmoving, a user does not move along a wall surface, but it is possiblefor the user to be stationary in a posture of leaning against a wall. Inview of this, one or more embodiments of the present invention, theperson detection unit also detects whether a person at a detectedposition is moving or stationary, and in a case of a detection point atwhich the person is stationary, the inferring unit sets thepredetermined distance lower than in a case of a detection point atwhich the person is moving.

Also, in one or more embodiments of the present invention, the inferringunit infers a placement region or the like of a furniture item or thelike as information regarding the room. For example, in a case where thepresence map includes a blank region that includes no detection pointsand is surrounded by detection points, the inferring unit may infer thatthe blank region is a region in which a furniture item is placed. Thisis because users cannot enter a region where a furniture item is placed,and therefore person detection is not performed in a furniture placementregion.

Furthermore, the person detection unit may also detect whether a personat a detected position is moving or stationary, and in a case where agroup of stationary points exists in a periphery of the blank region,the inferring unit may infer that a table and chairs are placed in theblank region. This is because a furniture item that users are oftenstationary around is typically a table or a chair. Note that in the casewhere a user sits in a chair, the head is detected at a low height.Accordingly, in one or more embodiments of the present invention, thecondition that there are many detection points with low head detectionheights is further added as a condition for inferring that a table isplaced in a blank region.

Also, in a case where the presence map includes a blank region thatincludes no detection points and is not surrounded by detection points,the inferring unit may infer that the blank region is a wall region or aregion in which a furniture item is placed next to a wall. In one ormore embodiments of the present invention, a separate sensor is used todetermine whether the blank region is a wall region or a region in whicha furniture item is placed.

Also, the inferring unit may infer a people containing region based onthe presence map, as information regarding the room. For example, theinferring unit can infer that, in the presence map, a region in whichthe number of detection points is higher than a predetermined percentageis a people containing region. Also, the person detection unit may alsodetect whether a person at a detected position is moving or stationary,and the inferring unit may infer that, in the people containing region,a region including more than a predetermined percentage of stationarydetection points is a stationary region. Alternatively, the inferringunit may infer that, in the people containing region, a region includingmore than a predetermined percentage of moving detection points is amovement region.

Also, it is possible to distinguish between a doorway in a room, astorage space, and the like based on the result of person trackingprocessing performed using person detection. For example, in one or moreembodiments of the present invention, the person detection unit alsoperforms processing for tracking a detected person, and in a case of alocation at which the number of intersections between person movementpaths and a boundary of the inferred room shape is greater than or equalto a predetermined number, the inferring unit infers that the locationis a doorway of the room or a storage space. This is because a locationon the boundary of the room shape where there are many intersectionswith movement paths is often a room doorway or a storage space such as acabinet or a closet. Here, in a case of a location at which the numberof intersections between person movement paths and the boundary of theinferred room shape is greater than or equal to a predetermined number,and at which person tracking can no longer be performed, or a person isnewly detected, the inferring unit may infer that the location is adoorway of the room. Also, in a case of a location at which the numberof intersections between person movement paths and a boundary of theinferred room shape is greater than or equal to a predetermined number,and at which person tracking can be continued, the inferring unit mayinfer that the location is a storage space. When a person exits adoorway, it is no longer possible to track that person, and when aperson enters through a doorway, person tracking starts at that time,and therefore the position of the doorway serves as the end point orstart point of person tracking. On the other hand, in the case of astorage space, person tracking can be continued. Due to this difference,it is possible to distinguish between a doorway and a storage space.

Also, if person detection results and time information are usedtogether, it is possible to infer a life scene. Specifically, the persondetection unit may also acquire time information indicating when theperson was detected, the room information inferring apparatus mayfurther include a storage unit that stores a life scene definition thatincludes a time period and a behavior pattern, and the inferring unitmay infer a life scene that appears in the room based on a behaviorpattern of a person obtained based on a detection result from the persondetection unit, time information indicating when the person wasdetected, and the life scene definition stored in the storage unit. Forexample, information indicating that if a predetermined behavior patternappears in a predetermined time period, a specific life scene isappearing is stored in the life scene definitions. Then if a definedbehavior pattern appears in a time period defined in the life scenedefinitions, it can be determined that the corresponding life scene isappearing in the room. In this case, it is also possible to specify thetime period in which and the location at which the life scene isactually appearing, for example. For example, it is possible to define alife scene in which, if a person is standing for a long duration at alocation in a certain region in the evening time period, the person iscooking at that location. If it is inferred by the life scene inferringunit that the “cooking” life scene is appearing, it is possible todetermine the time period in which cooking is actually taking place, thelocation of the kitchen, and the like.

Note that the present invention also encompasses a room informationinferring apparatus that includes at least a portion of the above units.Also, the present invention encompasses a room information inferringmethod, a computer program for causing a computer to execute the stepsof this method, and a non-transitory computer-readable storage mediumstoring this program. The present invention can be configured by anycombination of the above configurations and processes as long as notechnical conflict arises.

Also, the present invention encompasses an apparatus that performscontrol based on room information inferred using the room informationinferring apparatus or room information inferring method. For example,one or more embodiments of the present invention is an air conditioningapparatus that includes the above-described room information inferringapparatus and a control unit that performs air conditioning controlbased on room information inferred by the room information inferringapparatus. One or more embodiments of the present invention is alighting control apparatus that includes the above-described roominformation inferring apparatus and a control unit that performslighting control based on room information inferred by the roominformation inferring apparatus.

According to one or more embodiments of the present invention, it ispossible to infer information regarding a room simply and precisely,without input from a user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing functional blocks of an air conditionerapparatus that includes a room information inferring apparatus accordingto a first embodiment.

FIGS. 2A and 2B are external views of the air conditioner apparatus.

FIG. 3 is a flowchart showing a flow of air conditioning controlprocessing performed by the air conditioner apparatus.

FIGS. 4A and 4B are diagrams for describing a relationship between facedetection results and person positions.

FIGS. 5A to 5C are diagrams for describing a relationship between facedetection results and person positions.

FIG. 6 is a flowchart showing a flow of room information inferringprocessing performed by the room information inferring apparatus.

FIGS. 7A and 7B are diagrams showing an example of a person detectionresult.

FIG. 8 is a diagram for describing a room shape inferred based on aperson detection result.

FIGS. 9A to 9D are diagrams for describing processing for correcting aroom shape giving consideration to the distance between people and wallsurfaces.

FIG. 10 is a diagram for describing processing for detecting a furnitureplacement region in a room.

FIG. 11 is a flowchart showing a flow of processing for inferring adoorway and a storage space based on movement paths.

FIGS. 12A and 12B are diagrams for describing processing for inferring adoorway and a storage space based on peoples' movement paths.

FIG. 13 is a functional block diagram of an air conditioner apparatusthat includes a room information inferring apparatus according to asecond embodiment.

FIG. 14 is a flowchart showing a flow of life scene informationinferring processing according to the second embodiment.

DETAILED DESCRIPTION

Embodiments of the invention will be described below in detail in anillustrative manner with reference to the drawings. Note that unlessotherwise stated in particular, the dimensions, materials, shapes,relative arrangements, and the like of constituent parts described inthe following embodiments are not intended to limit the scope of thisinvention. In embodiments of the invention, numerous specific detailsare set forth in order to provide a more thorough understanding of theinvention. However, it will be apparent to one of ordinary skill in theart that the invention may be practiced without these specific details.In other instances, well-known features have not been described indetail to avoid obscuring the invention.

First Embodiment Configuration of Air Conditioning Apparatus

The following describes the overall configuration of an air conditioningapparatus (referred to below as “air conditioner apparatus”) accordingto a first embodiment of the present invention with reference to FIGS. 1and 2A and 2B. FIG. 1 is a diagram showing functional blocks of the airconditioner apparatus, and FIGS. 2A and 2B are external views of the airconditioner apparatus. An air conditioner apparatus 1 includes a roominformation inferring apparatus 2 in which a person detection functionis used, and carries out air conditioning control based on room-relatedinformation inferred by the room information inferring apparatus 2. Notethat in the first embodiment, a room shape and a furniture placementregion are inferred as room information, and air conditioning control iscarried out based on these pieces of room information.

The air conditioner apparatus 1 is mainly constituted by the roominformation inferring apparatus 2, a room information storage unit 3, anair conditioning control unit 4, a temperature sensor 5, a heatexchanger 6, and a blowing fan 7. The room information storage unit 3stores room shape information 3 a and furniture arrangement information3 b. The room information inferring apparatus 2 has a camera 21, aperson detection unit 22, a person position calculation unit 23, adetected position storage unit 24, and a room information inferring unit25.

The air conditioning control unit 4 is constituted by a processor, amemory, and the like, and, by the processor executing programs, the airconditioning control unit 4 determines operating conditions including adesired temperature, blowing direction, and blowing amount, and controlsthe heat exchanger 6, the blowing fan 7, and the like so as to operateunder the determined operating conditions. When determining theoperating conditions, the air conditioning control unit 4 givesconsideration to set conditions (e.g., set temperature) input from aremote control 8 and a room temperature obtained from the temperaturesensor 5, and additionally gives consideration to room-relatedinformation obtained from the room information inferring apparatus 2(the room shape information 3 a and the furniture arrangementinformation 3 b). Also, since the room information inferring apparatus 2has the person detection function, the air conditioning control unit 4also gives consideration to real-time person detection results whendetermining the operating conditions.

The temperature sensor 5 acquires the temperature in a room using aninfrared sensor, for example. The heat exchanger 6 is connected to anoutdoor unit (not shown) to configure a refrigerating cycle, and heatsand cools air taken into the heat exchanger 6. The blowing fan 7 is anapparatus that generates an air current, and circulates the air in aroom. The blowing amount and blowing direction (up and down directions,and left and right directions) of the blowing fan 7 can be adjusted.

The room information inferring apparatus 2 is an apparatus that capturesan overall image of a room, detects people in the captured image, andinfers room-related information based on the distribution of detectedpositions of persons. The room information inferring apparatus 2 of thefirst embodiment infers a room shape and furniture placement regions.The room information inferring apparatus 2 is configured as a computerthat includes a processor, a memory, and the like, and realizesfunctions of the person detection unit 22, the person positioncalculation unit 23, the detected position storage unit 24, the roominformation inferring unit 25, and the like by the processor executingprograms. Note that some or all of these functional units may beconstituted by an ASIC or FPGA circuit.

The camera 21 is attached to the front face of the air conditionerapparatus 1 as shown in FIG. 2A, and is constituted so as to be able tocapture an overall image of a room. As shown in FIG. 2B, the imagingdirection of the camera 21 is a direction facing downward by apredetermined angle from the direction perpendicular to the wall surfaceon which the air conditioner apparatus 1 is installed. The camera 21periodically performs imaging, and person detection processing iscarried out by the person detection unit 22 based on captured images.Although any imaging interval may be used, it is desirable that theinterval is short to a certain degree so as to be able to track people.The imaging interval is an interval of 1 second, for example.

The person detection unit 22 is a functional unit that detects a personin an image captured by the camera 21. The person detection unit 22 candetect a person by detecting a face in a captured image or parts of aface (e.g., eyes, nose, mouth). Alternatively, the person detection unit22 may detect a person by detecting a head or an upper body in acaptured image. The person detection unit 22 detects the position of aperson in a captured image, as well as the size of the face, head, orupper body. The person detection unit 22 may further detect thedirection of the face, body, or line of sight of a detected person, forexample.

The person position calculation unit 23 is a functional unit thatacquires the position of a person in a room based on the position andthe size of a face, a head, or an upper body in a captured image, whichare detected by the person detection unit 22. The person positioncalculation unit 23 calculates the three-dimensional position (a planeposition and a height) of the detected person based on the position andthe size of the face, the head, or the upper body in the captured image,and the angle of view, the installation angle, and the installationheight of the camera 21. The positions of persons calculated by theperson position calculation unit 23 are stored and accumulated in thedetected position storage unit 24. Note that time information indicatingwhen the person was detected is also stored as additional information inassociation with the positions. Other additional information is storedin association with detected positions, such as time informationindicating when imaging was performed, information indicating whetherthe detected person is standing or sitting, the movement path of thedetected person, and information indicating whether the person wasmoving or stationary when detected. Whether the detected person isstanding or sitting can be obtained based on the height of the detectedposition. Note that a lying position may be deemed to be a sittingposition, or these positions may be distinguished from each other, andit may be determined whether the detected person is standing, sitting,or lying down. The movement path can be acquired by person trackingprocessing. Also, whether the detected person is moving or stationarycan be detected based on the movement path. Hereinafter, a detectionpoint that is detected in a moving state is referred to as a movingpoint, and a detection point that is detected in a stationary state isreferred to as a stationary point.

The detected position storage unit 24 stores the positions (actualpositions) of persons in the room, which are calculated by the personposition calculation unit 23. Here, as previously described, thedetection time, the corresponding movement path, and informationindicating a moving point or a stationary point are also stored inassociation. By continuously performing person detection processing fora certain duration, information regarding multiple detection pointsdetected at different times is accumulated in the detected positionstorage unit 24.

The room information inferring unit 25 infers room information based onthe person detection results accumulated in the detected positionstorage unit 24. The room information inferring unit 25 infers the roomshape (the shape of wall surfaces), the furniture arrangement (furnitureplacement regions), the doorway placement positions, and the like basedon the distribution of person detection points. In the first embodiment,these pieces of information are collectively referred to as roominformation. The room information inferred by the room informationinferring unit 25 is stored in the room information storage unit 3 andreferenced when the air conditioning control unit 4 performs airconditioning control. A more detailed description of the content ofprocessing performed by the room information inferring unit 25 will bedescribed below with reference to flowcharts.

Processing Flow

Next, the flow of air conditioning control processing performed by theair conditioner apparatus 1 will be described with reference to theflowcharts of FIGS. 3 and 6. FIG. 3 is a flowchart showing the overallflow of air conditioning control processing, and FIG. 6 is a flowchartshowing the flow of room information inferring processing.

First, the camera 21 captures an image of the room in which the airconditioner apparatus 1 is installed (the room to be subjected to roominformation inferring) (step S10), and the person detection unit 22performs person detection on the captured image (step S11). FIG. 4Ashows an example of a person detection result. In FIG. 4A, three persons41 a, 41 b, and 41 c have been detected. The person detection unit 22acquires the positions of the detected persons in the image, and thesizes of the persons (the sizes of faces, since faces have been detectedin this case).

The person position calculation unit 23 specifies the positions of thedetected persons in the room based on the positions of the detectedpersons in the image and the sizes of the detected persons, as well asthe angle of view, the installation height, and the like of the camera21 (step S12). FIG. 4B shows plane positions 42 a, 42 b, and 42 c in theroom, which correspond to the detected persons 41 a, 41 b, and 41 c inFIG. 4A. Since the directions in which the persons are present are knownbased on the positions of the persons in the captured image, and theirdistances from the camera 21 are known based on the sizes of the persons(the faces in this case), it is possible to calculate three-dimensionalpositions of the persons. FIGS. 5A to 5C show examples of the case ofdetecting persons that are at approximately the same horizontal positionand have different heights. FIG. 5A shows an image captured by thecamera 21 in this situation. In FIG. 5A, two persons 43 and 44 have beendetected. It is understood that the person 44 is at a low position basedon the fact that the size of their face is relatively large regardlessof being located at a lower position in the captured image. In otherwords, based on the positions and sizes in the image, the positionswhere the persons 43 and 44 are present can be determined to be thelocations shown in FIGS. 5B and 5C.

The person position calculation unit 23 stores the positions of thepersons (detected positions), which were calculated in step S12, in thedetected position storage unit 24. At this time, the detection times(imaging times) are also stored in the detected position storage unit 24along with the detected positions. Also, whether the detected personsare standing or sitting is determined based on the height informationincluded in the detected positions, and information indicating thedetermination results is stored in the detected position storage unit 24in association with the detection points.

The processing from step S10 to step S13 is executed repeatedly.Accordingly, the detected positions of multiple detection results areaccumulated in the detected position storage unit 24. Also, as persondetection is executed repeatedly, the person position calculation unit23 also performs processing for tracking the movement paths of persons.The acquired movement paths are stored in the detected position storageunit 24, and information indicating the movement paths that therespective detection points correspond to is also stored in the detectedposition storage unit 24. Also, information indicating whether thedetected persons are moving or stationary is also determined based onthe movement paths and stored. For example, if the speed of movementobtained based on a movement path is greater than or equal to apredetermined threshold value, it can be determined that thecorresponding person is moving, and if this speed is less than thethreshold value, it can be determined that the corresponding person isstationary.

When the processing from step S10 to step S13 has been repeated for apredetermined duration, the room information inferring unit 25 infersroom information based on the information accumulated in the detectedposition storage unit 24 (step S14). A detailed flow of room informationinferring processing is shown in the flowchart of FIG. 6.

In step S601, the room information inferring unit 25 acquiresinformation regarding accumulated person detection points from thedetected position storage unit 24, and generates a detection pointdistribution (presence map). In presence map generation processing, amap plotting the plane positions included in the person detectedpositions is generated.

The following description takes the example of performing persondetection for a certain duration in a room 70 shown in FIG. 7A. A table71 is placed in the center of the room 70, and chairs 72 a to 72 f areplaced around the table 71. Also, a bookshelf (storage furniture item)73 is placed in the room 70 in the lower right corner of the figure, anda door (doorway) 74 is placed in the lower left corner of the figure.The air conditioner apparatus 1 of the first embodiment is installed onthe wall surface of the room located on the upper side in the figure,and camera imaging and person detection are performed in this state.

FIG. 7B shows a presence map 75 indicating the distribution of thedetection points that were detected in the above example. The positions(horizontal positions) of crosses in FIG. 7B indicate that persons weredetected.

First, the room information inferring unit 25 infers the shape of theroom. Room shape inferring processing includes the processing of stepS602 and step S603. In step S602, the room information inferring unit 25calculates a polygon that is circumscribed around the distribution ofperson detection points in the presence map, and infers that thecalculated polygon is the shape of the room (the shape of wallsurfaces). The polygon may be any polygon, but since rooms generallyhave a rectangular (elongated rectangular) shape, the room informationinferring unit 25 can infer that the shape of the room is a rectanglecircumscribed around the distribution of person detection points. Here,it is assumed that the sides of the rectangle are parallel to thedirection of the wall surface on which the air conditioner apparatus 1is installed, and the direction orthogonal to this direction (which arethe same as the horizontal imaging direction of the camera 21, and thehorizontal direction orthogonal to this direction).

FIG. 8 shows the room shape obtained based on the presence map 75 shownin FIG. 7B. Here, a room shape 80 is obtained as the smallestcircumscribed rectangle around the detection points in the presence map75.

Note that the room shape inferred based on the presence map need not belimited to a rectangle, and it is sufficient that a circumscribedpolygon is inferred as the room shape. Here, in one or more embodimentsof the present invention, the polygon circumscribed around the presencemap is obtained based on the assumption that the room is defined bystraight lines (walls) that are parallel to two directions, namely thedirection of the wall surface on which the air conditioner apparatus 1is installed, and the direction orthogonal to this direction (i.e., thehorizontal imaging direction of the camera 21 and the horizontaldirection orthogonal to this direction). This is because although a roomdoes not have a rectangular shape if it includes a column or the like,the directions of the respective wall surfaces are generally twodirections that are orthogonal to each other.

Next, the room information inferring unit 25 subjects the room shapeinferred in step S602 to correction processing for correcting wallsurface positions (the room shape) (step S603). In step S602, arectangle circumscribed around person detection points is obtained asthe room shape. However, it is rare for people to actually be located atpositions in contact with a wall. In general, as shown in FIG. 9A, it isthough that a person 91 is located at a position separated from a wallsurface 92 by a certain distance 93. In particular, in the case wherethe person 91 is moving, the distance between the person 91 and the wallsurface 92 increases. In view of this, the room information inferringunit 25 infers the shape of the room based on the assumption that thewall surfaces in the actual room shape are outward by a predetermineddistance from the circumscribed rectangle 80 that was inferred in stepS602. However, it is conceivable that the person 91 is stationary andleaning against the wall surface 92, and in this case the distance 93 tothe wall surface is lower than in the case of a moving point (thedistance 93 may be deemed to be zero). In view of this, it is sufficientthat in the case where the detection point is a moving point, thepredetermined distance is set higher, whereas in the case where thedetection point is a stationary point, the predetermined distance is setlower.

Take the example where a room shape 94 is inferred in step S602 based onthe detection point distribution shown in FIG. 9B. Note that in FIG. 9B,black circles represent moving points, and white circles representstationary points. The room information inferring unit 25 extractsmoving points that are located in the vicinity of boundaries of theinferred room shape 94, and it is deemed that detection points existoutward of the actual detection points. As shown in FIG. 9C, in the caseof a moving point in the vicinity of a wall surface, it is deemed that adetection point exists at a position separated by a predetermineddistance in a direction orthogonal to the nearby wall surface (circlewith a cross). In the case of a detection point in the vicinity of acorner of the room, a deemed detection point is set for each of the wallsurfaces making up the corner. Also, in the case of a stationary point,this processing is not performed, or it is deemed that a detection pointexists at a position separated by a shorter distance than in the case ofa moving point (in FIG. 9C, processing for setting a deemed detectionpoint is not performed in the case of stationary points). The roominformation inferring unit 25 can acquire a room shape 95 shown in FIG.9C by re-obtaining a rectangle circumscribed around detection pointsthat include the deemed detection points set as described above. Theroom shape 95 obtained in this way can be said to be a shape that ismore accurate to the extent that consideration is given to the distancebetween wall surfaces and people.

Note that although a rectangle circumscribed around the detection pointsis obtained two times in the above example, the processing of step S602and step S603 can be performed in a collective manner. As shown in FIG.9D, moving points are deemed to have a spread of a predetermineddistance. The predetermined spread is indicated by dashed-line circles.Stationary points are deemed to not have a spread, or are deemed to havea spread with a smaller radius than the moving points. By obtaining arectangle circumscribed around detection points having these spreads, itis possible to obtain the same room shape 95 as above.

Next, the room information inferring unit 25 detects the placementregion of furniture items in the room. Furniture placement regioninferring processing includes the processing from step S604 to step S607described below. Specifically, first, in step S604, the room informationinferring unit 25 determines whether the presence map includes a regionin which no person detection points are located (called a blank region),and if such a blank region is included, specifies the range of the blankregion. In the example in FIG. 8, regions not including any persondetection points are located in the center and the lower right portionof the figure, and therefore two blank regions 1001 and 1002 arespecified as shown in FIG. 10. A rectangle inscribed by surroundingdetection points may be obtained as the position of a blank region, orthe position of a blank region may be determined such that severaldetection points are located inside the blank region. This is becausethere are cases where a person's head is located above a furniture item.Also, the blank region is not limited to having a rectangular shape, andit may have another polygonal shape, or any other shape such as acircular or elliptical shape.

When blank regions are specified, for each blank region, the roominformation inferring unit 25 determines whether the blank region issurrounded by person detection points (step S605). A blank regionsurrounded by person detection points (YES in step S605) is inferred tobe a table (step S606), and a blank region not surrounded by persondetection points (NO in step S605) is inferred to be a wall region or afurniture item placed next to a wall (step S607). For example, since theblank region 1001 in FIG. 10 is surrounded by person detection points,it can be inferred that a table is placed in that region. On the otherhand, since the blank region 1002 is not surrounded by person detectionpoints, it can be inferred that the region is a wall region or afurniture item placed next to the wall.

Furthermore, in step S608, the room information inferring unit 25 infersadditional information regarding the room shape based on the personmovement paths obtained by person tracking processing. Examples of theadditional information include the locations of doorways, the locationsof storage furniture items and the like, and the locations of aisles.

A technique for inferring the locations of doorways and storagefurniture items will be described below with reference to FIGS. 11 and12. The room information inferring unit 25 acquires the room shapeinformation that was inferred through the processing up to step S607,and specifies the boundary of the room (step S1101). The boundary of theroom shape refers to the substantial boundary of the habitable space inthe room, and therefore refers to the boundary of the region that doesnot include furniture items placed next to the walls. It is assumed thata room shape 1201 shown in FIG. 12A has been inferred through theprocessing up to step S607. Also, a region 1202 is inferred to be ablank region not surround by person detection points, that is to say awall region or a furniture item placed next to a wall. In this case, aboundary 1203 of a region excluding the region 1202 from the room shape1201 is specified as the boundary of the room shape. Note that theboundary of the room shape may be determined as the perimeter in thepresence map.

Next, the room information inferring unit 25 acquires the personmovement paths (the results of person tracking processing) stored in thedetected position storage unit 24 (step S1102). FIG. 12B is a diagramshowing an example of movement paths, and the movement paths areindicated by lines with arrow heads. Note that crosses indicatelocations where person tracking could no longer be performed, andcircles indicate locations where person tracking started (where a personwas newly detected).

The room information inferring unit 25 specifies a location where thenumber of intersections between movement paths and the boundary 1203 isgreater than or equal to a predetermined number (step S1103). In theexample in FIG. 11B, the two regions 1204 and 1205 can be specified. Thepredetermined number referred to here may be a certain prescribednumber, or may be a number determined according to the number ofmovement paths stored in the detected position storage unit 24, forexample.

The room information inferring unit 25 determines whether or not persontracking could be continued at the location specified in step S1103(step S1104). A location where person tracking could not be continued(NO in step S1104) is inferred to be a location where a doorway isplaced (step S1105), and a location where person tracking could becontinued (YES in step S1104) is inferred to be a storage space (stepS1106). If a person enters a doorway, that person is newly detected, andif a person exits the doorway, it is no longer possible to performperson tracking at that time. On the other hand, even if a personapproaches a storage space such as a closet or a cabinet, they do notexit the room, and therefore person tracking can be continued.Accordingly, it is possible to make the above-described determination.

Note that the example of inferring the placement locations of a doorwayand a storage space is described as an example of the additionalinformation inferring of step S1308. However, other information may beacquired as additional information. For example, a location where peoplemove and do not remain stationary (movement region) can be determined tobe an aisle. Also, a location where people often remain stationary(stationary region) can be determined to be a table, a sofa, a bed, akitchen, or the like. Furthermore, it is possible to determine which ofthe above the location corresponds to based on the detected height ofthe stationary people. It is also possible to determine whether thelocation corresponds to a table or a sofa based on the line of sightdirections or face directions at the stationary points. In the case of atable, people often sit facing each other, whereas in the case of asofa, people rarely sit facing each other, and this makes it possible todistinguish between the two. In this way, not only the shape of theroom, but also various types of additional information can be acquiredbased on person detection results.

The room information inferring unit 25 outputs information such as theshape of the room, furniture placement locations, and doorway locations,which were obtained as described above, and stores this information inthe room information storage unit 3. Room information inferringprocessing performed in step S14 of FIG. 3 has been described above.

In step S15, the air conditioning control unit 4 carries out airconditioning control based on the inferred room information. Forexample, the blowing amount and the blowing direction of the blowing fan7 are appropriately controlled based on the shape of the room.Specifically, in the case where the distance to the opposing wall islong, the blowing amount is increased, and the blowing direction isadjusted so as to circulate the air and prevent the formation of hotspots in the room. It is also possible to perform control such that theair stream does not directly hit a location where a person isstationary, and perform control for conversely causing an air stream todirectly hit a location where a person is stationary. If the type oflocation where the person is stationary is known (e.g., a table, a sofa,a bed, or a kitchen), control can be performed in accordance with thetype of location. For example, it is conceivable to perform control forpreventing an air stream from directly hitting a table, a sofa, and abed, but causing an air stream to directly hit a kitchen. Also, in oneor more embodiments of the present invention, if the location of adoorway or an aisle is known, air conditioning control is carried outgiving consideration to this information as well. Note that the airconditioning control unit 4 can perform air conditioning control givingconsideration to not only room information that has been inferred inadvance, but also information such as the number and locations of peopleacquired in real-time by the person position calculation unit 23.

According to the air conditioner apparatus of the first embodimentdescribed above, even if a user does not input information regarding aroom, room information can be automatically inferred by the roominformation inferring apparatus. Since a technique for detecting aperson based on a face, a head, or an upper body is used in this roominformation processing, an image captured by a normal monocular camerais sufficient, thus eliminating the need for an expensive camera such asa stereo camera or a TOF camera, and making it possible to suppress theapparatus manufacturing cost. Furthermore, the three-dimensionalposition (particularly the height) of a person is detected, and whetherthe person is moving or stationary is also detected based on theirmovement path. Taking these pieces of information into considerationmakes it possible to determine the categories of locations in the room.Since the shape of the room and the categories of locations therein areknown, the air conditioner apparatus can carry out appropriate airconditioning control in accordance with the room in which it isinstalled.

Variation of First Embodiment

Although the shape of the room is inferred in the first embodiment, itis not necessarily required to infer the shape of the room asroom-related information. For example, a people containing region may beinferred based on a presence map plotting person detection points. Evenif the shape of the room is not known, if a people containing region isknown, appropriate air conditioning control can be carried out inaccordance with such information.

A people containing region can be inferred as described below, forexample. The room information inferring unit 25 specifies a region inthe presence map that includes a predetermined percentage or more ofperson detection points (this region can also be said to be a region inwhich the density of person detection points is high), and infers thatthe specified region is a people containing region. In other words, alocation with a high probability of people being present is inferred tobe a people containing region.

Note that if the person detection unit performs person trackingprocessing, and it is known whether the respective detection points aremoving points or stationary points, the people containing region can bedivided into smaller regions. Specifically, within the people containingregion, a region containing many stationary points is inferred to be astationary region. A stationary region is a location where people remainfor a relatively long time, such as a table or sofa, for example.Conversely, within the people containing region, a region containingmany moving points is inferred to be a movement region. A movementregion is a region used by people solely for movement, such as an aisle.Note that whether or not a region includes many stationary points ormoving points can be determined based on whether or not a predeterminedpercentage or more of all of the detection points in the region arestationary points or moving points.

The use of a people containing region makes it possible to perform airconditioning control in which, for example, a people containing region(particularly a stationary region) is cooled with priority in accordancewith the user needs. Also, air conditioning control can be performedsuch that an air stream does not directly hit a people containing region(particularly a stationary region). In this way, even if the room shapeis not known, if the people containing region is known, air conditioningcontrol that achieves comfort and energy reduction can be performed.

Second Embodiment

An air conditioner apparatus of the second embodiment has theconfiguration of the first embodiment, and additionally infers a lifescene lifecycle based on person detection results, and carries outappropriate air conditioning control giving consideration to life scenesin addition to the room shape.

A life scene is a typical behavior pattern that arises in a livingspace. Examples of life scenes include behavior patterns such ascooking, eating, spending family time together, and studying. A lifescene lifecycle is a cycle indicating the time periods in whichrespective life scenes appear. The appropriate air conditioning controlmethod differs according to the life scene, and therefore if the lifescene lifecycle is known, favorable air conditioning control can berealized using the life scene lifecycle.

FIG. 13 is a functional block diagram of the air conditioner apparatus 1according to the second embodiment. A comparison with the firstembodiment shows differences in that the room information inferringapparatus 2 includes a life scene rule storage unit 26, and the roominformation inferring unit 25 infers life scene information as well.Also, the room information storage unit 3 stores life scene information3 c as well, and air conditioning control is performed givingconsideration to the life scene information 3 c as well.

The life scene rule storage unit 26 stores rules defining life scenes. Alife scene includes information indicating the time period in which thelife scene appears, and the behavior pattern that occurs therein. Thefollowing example shows life scene rules for two life scenes, namely“cooking” and “dinner time”.

Life scene name: Cooking

Time period: evening to night

Behavior pattern: movement while standing for a long duration (e.g., 30minutes or more) in a specific region (kitchen)

Life scene name: Dinner time

Time period: evening to night

Behavior pattern: stationary while sitting at a specific location(dining table). In particular, multiple people are stationary andsitting facing each other.

The above rules are relatively simple rules, and more detailed rules maybe described. Also, besides the life scenes given as examples above,rules regarding more life scenes are stored in the life scene rulestorage unit 26.

The room information inferring unit 25 determines the behavior patternthat corresponds to the life scene rules based on the person detectionresults stored in the detected position storage unit 24 in addition tothe room information described in the first embodiment. Note that sinceinformation indicating the times when detection points were detected isneeded in the second embodiment, the detected position storage unit 24needs to store the positions of person detection points along with timeinformation indicating when the detection points were detected.

Air conditioning control processing and room information inferringprocessing of the second embodiment is basically the same as in thefirst embodiment (FIGS. 3 and 6). Note that the second embodiment isdifferent in that life scene information inferring processing is addedin room information inferring processing. FIG. 14 is a flowchart showingthe flow of life scene information inferring processing. This life sceneinferring processing is performed after the processing of step S608 inroom information inferring processing, for example.

First, the room information inferring unit 25 selects one life scene andacquires the rule thereof from the life scene rule storage unit 26 (stepS1501). Next, the room information inferring unit 25 references thedetected position storage unit 24 and determines whether or not acurrent behavior pattern matches the selected rules (step S1502). Forexample, in the case of the “cooking” life scene, it is determinedwhether or not a current behavior pattern is the behavior pattern ofmoving while standing for a long duration in a specific region in theevening time period. Also, in the case of the “dinner time” life scene,it is determined whether or not a current behavior pattern is astationary pattern of being stationary while sitting in the time periodfrom evening to night.

If the person detection results include a behavior pattern that matchesthe life scene rule (YES in step S1502), the room information inferringunit 25 generates life scene information indicating that the life scene(behavior pattern) appeared, and the time period in which and locationat which the life scene actually appears (step S1503). For example, theobtained information indicates that the “cooking” life scene appearsfrom 6 o'clock PM to 7 o'clock PM, and the location of the region(kitchen) where that behavior pattern appears. Alternatively, theobtained information indicates that the “dinner time” life scene appearsfrom 7 o'clock PM to 8 o'clock PM, and the location of the region(dining table) where that behavior pattern appears.

Processing for one life scene is completed through the processingdescribed above. The room information inferring unit 25 then determineswhether an unprocessed life scene exists (step S1504), and if anunprocessed life scene exists (YES in step S1504), the procedure returnsto step S1501, and processing is performed for the next life scene. Onthe other hand, if processing has been completed for all life scenes (NOin step S1504), the generated life scene information is output (stepS1505). The life scene information is stored in the room informationstorage unit 3 as part of the room information.

Lastly, air conditioning control that employs life scene informationwill be described briefly. If the above life scene information is known,the air conditioning control unit 4 can realize even more favorable airconditioning control. For example, in the “cooking” life scene, it ishot in the kitchen during cooking, and therefore it is possible toperform control for causing a relatively strong air stream to directlyhit a person. In this case, the life scene information includesinformation indicating the time period in which the “cooking” life sceneappears, and the location of the kitchen, thus making it possible torealize control for directing an air stream toward the location of thekitchen in that time period. Similarly, in the “dinner time” life scene,it is not comfortable to be directly hit with an air stream, andtherefore it is possible to realize control for directing an air streamtoward the periphery of the table.

According to the second embodiment, it is possible to acquireinformation regarding life scenes that appear at the location where theair conditioner apparatus is installed based on person detectionresults, and it is possible to realize appropriate air conditioningcontrol that gives consideration to life scenes. The user does not needto directly input information regarding life scenes here, and this isvery user-convenient.

Other Embodiments

The description of the above embodiments merely describes embodiments ofthe present invention by way of illustrative examples, and the presentinvention is not intended to be limited to the above specificembodiments. The present invention can be modified in various wayswithin the scope of the technical idea of the invention.

In the first and second embodiments, the room information inferringapparatus infers various types of information as room information, andthe air conditioner apparatus performs air conditioning control based onthe inferred information. Various modifications are possible withrespect to the room information that is inferred and used in airconditioning control. For example, although room information such as theroom shape, the furniture arrangement, doorways, storage furnitureitems, and stationary regions and movement regions of people (peoplecontaining regions) is acquired in the first embodiment, as long asinformation includes at least some of the above, that information can bemade use of in air conditioning control. Accordingly, the roominformation inferring apparatus may infer at least a portion of thesetypes of information. Also, although air conditioning control isperformed using a combination of information including life sceneinformation and room shape information in the second embodiment, it ispossible to similarly use any room information other than life sceneinformation. For example, air conditioning control can be performedbased on simply life scene information and furniture arrangementinformation, and air conditioning control can be performed based onsimply life scene information and people containing regions.

In the above descriptions, the room information inferring apparatus 2 isbuilt into the air conditioner apparatus 1. However, the roominformation inferring apparatus 2 may be constituted as an apparatusseparate from the air conditioner apparatus 1. As long as roominformation and life scene information inferred by the room informationinferring apparatus 2 is transmitted to the air conditioner apparatus bywireless communication or the like, the air conditioner apparatus canrealize air conditioning control based on such information.

Also, although an air conditioner apparatus (cooling and heating device)is described as an example of the air conditioning apparatus, the roominformation inferring apparatus may be incorporated in an apparatus suchas an air purifier, a humidifier, or a blower. Also, the roominformation inferring apparatus can be incorporated in and used in anyapparatus other than an air conditioning apparatus, as long as it is anapparatus in which optimal control is determined according to roominformation. For example, it is conceivable for the room informationinferring apparatus to be incorporated in and used in a lighting controlapparatus or the like.

While the invention has been described with respect to a limited numberof embodiments, those skilled in the art, having benefit of thisdisclosure, will appreciate that other embodiments can be devised whichdo not depart from the scope of the invention as disclosed herein.Accordingly, the scope of the invention should be limited only by theattached claims.

The invention claimed is:
 1. A room information inferring apparatus thatinfers information regarding a room, comprising: an imaging unit thatcaptures an image of a room that is to be subjected to inferring; aperson detector that detects a person in an image captured by theimaging unit, and acquires a position of the person in the room; apresence map generator that generates a presence map indicating adistribution of detection points corresponding to persons detected in aplurality of images captured at different times; and an inferring unitthat infers information regarding the room based on the presence map,wherein, in a case where the presence map includes a blank region thatincludes no detection points and is surrounded by detection points, theinferring unit infers that the blank region is a region in which afurniture item is placed.
 2. The room information inferring apparatusaccording to claim 1, wherein the person detector detects a face, ahead, or an upper body of the person in the image, and acquires theposition of the person in the room based on a position and a size of theface, the head, or the upper body in the image.
 3. The room informationinferring apparatus according to claim 1, wherein the inferring unitinfers a shape of the room based on the presence map.
 4. The roominformation inferring apparatus according to claim 3, wherein theinferring unit infers that a polygon circumscribed around thedistribution of detection points in the presence map is the shape of theroom.
 5. The room information inferring apparatus according to claim 4,wherein the inferring unit infers the shape of the room based on anassumption that the room is defined by straight lines that are parallelin two mutually orthogonal directions.
 6. The room information inferringapparatus according to claim 5, wherein the two directions arerespectively an imaging direction of the imaging unit and a directionorthogonal to the imaging direction.
 7. The room information inferringapparatus according to claim 4, wherein, when inferring the shape of theroom, the inferring unit deems that a wall surface is outward by apredetermined distance from a shape obtained as a polygon circumscribedaround the distribution of detection points in the presence map.
 8. Theroom information inferring apparatus according to claim 7, wherein theperson detector also detects whether a person at a detected position ismoving or stationary, and wherein, in a case of a detection point atwhich the person is stationary, the inferring unit sets thepredetermined distance lower than in a case of a detection point atwhich the person is moving.
 9. The room information inferring apparatusaccording to claim 1, wherein the person detector also detects whether aperson at a detected position is moving or stationary, and wherein, in acase where a group of stationary points exists in a periphery of theblank region, the inferring unit infers that a table is placed in theblank region.
 10. The room information inferring apparatus according toclaim 1, wherein in a case where the presence map includes a blankregion that includes no detection points and is not surrounded bydetection points, the inferring unit infers that the blank region is awall region or a region in which a furniture item is placed next to awall.
 11. The room information inferring apparatus according to claim 1,wherein the inferring unit infers a people containing region based onthe presence map.
 12. The room information inferring apparatus accordingto claim 11, wherein the person detector also detects whether a personat a detected position is moving or stationary, and wherein theinferring unit infers that, in the people containing region, a regionincluding more than a predetermined percentage of stationary detectionpoints is a stationary region.
 13. The room information inferringapparatus according to claim 11, wherein the person detector unit alsodetects whether a person at a detected position is moving or stationary,and wherein the inferring unit infers that, in the people containingregion, a region including more than a predetermined percentage ofmoving detection points is a movement region.
 14. The room informationinferring apparatus according to claim 1, wherein the person detectoralso performs processing for tracking a detected person, and wherein, ina case of a location at which the number of intersections between personmovement paths and a boundary of the inferred room shape is greater thanor equal to a predetermined number, the inferring unit infers that thelocation is a doorway of the room or a storage space.
 15. The roominformation inferring apparatus according to claim 14, wherein, in acase of a location at which the number of intersections between personmovement paths and the boundary of the inferred room shape is greaterthan or equal to a predetermined number, and at which person trackingcan no longer be performed, or a person is newly detected, the inferringunit infers that the location is a doorway of the room.
 16. The roominformation inferring apparatus according to claim 14, wherein, in acase of a location at which the number of intersections between personmovement paths and a boundary of the inferred room shape is greater thanor equal to a predetermined number, and at which person tracking can becontinued, the inferring unit infers that the location is a storagespace.
 17. The room information inferring apparatus according to claim1, wherein the person detector also acquires time information indicatingwhen the person was detected, wherein the room information inferringapparatus further comprises a storage unit that stores a life scenedefinition that includes a time period and a behavior pattern, andwherein the inferring unit infers a life scene that appears in the roombased on a behavior pattern of a person obtained based on a detectionresult from the person detector, time information indicating when theperson was detected, and the life scene definition stored in the storageunit.
 18. A room information inferring method carried out by a roominformation inferring apparatus, comprising: an imaging step ofcapturing an image of a room that is to be subjected to inferring; aperson detection step of detecting a person in an image captured in theimaging step, and acquiring a position of the person in the room; apresence map generation step of generating a presence map indicating adistribution of detection points corresponding to persons detected in aplurality of images captured at different times; and an inferring stepof inferring information regarding the room based on the presence map,wherein, in a case where the presence map includes a blank region thatincludes no detection points and is surrounded by detection points, theinferring apparatus infers that the blank region is a region in which afurniture item is placed.
 19. A program stored on a non-transitorycomputer readable medium that causes a computer to execute a roominformation inferring method comprising: an imaging step of capturing animage of a room that is to be subjected to inferring; a person detectionstep of detecting a person in an image captured in the imaging step, andacquiring a position of the person in the room; a presence mapgeneration step of generating a presence map indicating a distributionof detection points corresponding to persons detected in a plurality ofimages captured at different times; and an inferring step of inferringinformation regarding the room based on the presence map, wherein, in acase where the presence map includes a blank region that includes nodetection points and is surrounded by detection points, the inferringmethod infers that the blank region is a region in which a furnitureitem is placed.
 20. An air conditioning apparatus comprising: the roominformation inferring apparatus according to claim 1; and a controllerthat performs air conditioning control based on information regarding aroom that is inferred by the room information inferring apparatus.